Self-driving vehicle system with retractable sensor head

ABSTRACT

A self-driving vehicle system comprising a body having one or more motorized wheels, and a sensor head coupled to the body. The sensor head is movable from a retracted position to an extended position relative to the body. The sensor head comprises one or more proximity sensors and one or more cameras.

BACKGROUND Field

Embodiments disclosed herein relate to a self-driving vehicle systemwith retractable sensor head.

Description of the Related Art

Automated guided vehicles (AGVs) are autonomous self-driving vehiclesused in a variety of different environments. For example, AGVs are usedin warehouses to assist with moving inventory from one area to another.However, one problem that operators face is when there is a change inthe arrangements or heights of the shelves that store such inventory,the AGVs have to be taken offline and reprogrammed to account for suchchanges. Another problem is that the AGVs cannot detect when inventoryis placed on the AGV or even the quantity or type of inventory. Afurther problem is that the AGVs cannot detect obstacles located behindthe AGV when moving in reverse. These problems often result in areduction in productivity and efficiency, as well as potential safetyhazards.

Therefore, there exists a need for new and improved self-driving vehiclesystems.

SUMMARY

In one embodiment, a self-driving system comprises a body having one ormore motorized wheels; and a sensor head coupled to the body and movablefrom a retracted position to an extended position relative to the body,wherein the sensor head comprises one or more proximity sensors.

In one embodiment, a self-driving system comprises a body having one ormore motorized wheels; and a sensor head coupled to the body, whereinthe sensor head comprises one or more side facing cameras, rear facingcameras, or down facing cameras.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a side view of an automated guided vehicle (AGV) having aretractable sensor head according to one embodiment.

FIG. 2 is another side view of the AGV according to one embodiment.

FIG. 3 is a top view of the AGV according to one embodiment.

FIG. 4 is a schematic view of the AGV detecting an object located behindthe AGV according to one embodiment.

FIG. 5 is a schematic view of the AGV detecting and displaying inventoryinformation according to one embodiment.

FIG. 6 is a schematic view of the AGV following an object located andmoving behind the AGV while the AGV is moving in reverse according toone embodiment.

FIG. 7 is an exploded view of a housing of the sensor head of the AGVaccording to one embodiment.

FIG. 8 is a block diagram of the AGV according to one embodiment.

To facilitate understanding, identical reference numerals have beenused, where possible, to designate identical elements that are common tothe figures. It is contemplated that elements disclosed in oneembodiment may be beneficially utilized with other embodiments withoutspecific recitation.

DETAILED DESCRIPTION

Embodiments of the disclosure include self-driving systems having one ormore shelf detection proximity sensors, rear detection proximitysensors, side facing cameras, rear facing cameras, down facing cameras,and any combination of these sensors and cameras. Automated guidedvehicles (AGVs) are self-driving vehicles that include but are notlimited to mobile robots, such as autonomously-navigating mobile robots,inertially-guided robots, remote-controlled mobile robots, and/or robotsguided by laser targeting, vision systems, and/or roadmaps. Although theembodiments of the self-driving systems are described and illustratedherein with respect to AGVs moving inventory in a warehouse environment,the embodiments may be used with any type of self-driving systems in anytype of environment.

FIG. 1 is a side view of an automated guided vehicle (AGV) 100 accordingto one embodiment. The AGV 100 includes a body comprising a console 30coupled in an upright positon to a front end of a mobile base 20. Theconsole 30 has a display 50 configured to display information and/orallow an operator 200 to control the operation of the AGV 100. Themobile base 20 has a plurality of motorized wheels 40 configured torotate and/or roll in any given direction to move the AGV 100. Themobile base 20 has an upper surface 25 that can be used to supportinventory 65. The AGV 100 is configured to move inventory 65 to and fromdifferent locations, such as to and from a storage rack 80 having one ormore shelves 85.

The AGV 100 includes a sensor head 10 that is shown coupled to theconsole 30 by one or more extendable/retractable rods 35 configured tomove the sensor head 10 between a retracted position and an extendedposition as shown by reference arrow A. The sensor head 10 is movable bythe extendable/retractable rods 35 in both the vertical direction andthe horizontal direction relative to the console 30 and the mobile base20. The sensor head 10 can be extended and moved to a higher position(or retracted and moved to a lower position) to prevent inventory orother objects from obstructing sensors and cameras coupled to the sensorhead 10 during operation. The sensor head 10 can also be moved higher orlower to detect the location of one or more shelves of different storageracks.

FIG. 2 is another side view of the AGV 100. The sensor head 10 of theAGV 100 may include one or more rear facing cameras 11, one or more sidefacing cameras 12, one or more down facing cameras 13, one or more reardetection proximity sensors 14, and/or one or more shelf detectionproximity sensors 15. Any number or arrangement of side, rear, and downfacing cameras 11, 12, 13, as well as any number or arrangement of reardetection and shelf detection proximity sensors 14, 15 can be used withthe AGV 100. The rear detection proximity sensors 14 and/or the shelfdetection proximity sensors 15 are configured to detect the presence ofnearby objects and may include sonar sensors, infrared sensors, radarsensors, and/or LiDAR sensors.

The rear detection proximity sensors 14 are configured to detect objectslocated behind the AGV 100, such as one or more operators 200 and/or oneor more inventory 65. For example, the rear detection proximity sensors14 can be used to help prevent the AGV 100 from inadvertently contactingan object located behind the AGV 100, such as by avoiding an obstaclewhen the AGV 100 is moving in reverse. The shelf detection proximitysensors 15 are configured to detect a location of the shelf 85 and/or alocation of an object on the shelf 85. For example, the sensor head 10can be raised and lowered by the extendable/retractable rods 35 untilthe shelf detection proximity sensors 15 detect the location of theshelf 85 and/or the location of an object on the shelf 85

The rear facing cameras 11 are configured to detect and record an imagesof objects located behind the AGV 100, such as one or more markers 75,one or more operators 200, and/or one or more inventory 65. The sidefacing cameras 12 are configured to detect and record images of objectslocated on the sides of the AGV 100, such as one or more markers 75, oneor more operators 200, and/or one or more inventory 65. The down facingcameras 13 are configured to detect and record images of objects locatedon the AGV 100, such as one or more markers 70 and/or one or moreinventory 65. The markers 70, 75 can be barcodes and attached to anyobject, such as the storage rack 80, the shelf 85, and/or the inventory65.

As shown in FIG. 3, the side, rear, and down facing cameras 11, 12, 13are located on the AGV 100 to provide an image capturing range 16 thatincludes areas slightly in the front of the AGV 100, on both sides ofthe AGV 100, and behind the AGV 100. The image capturing range 16 mayinclude a 180 degree viewing area, a 270 degree viewing area, a 360degree viewing area, or any viewing area between 180 degrees and 360degrees. The side, rear, and down facing cameras 11, 12, 13 areconfigured to detect and record images of nearby objects. The cameras 30may include but are not limited to a monocular camera, a binocularcamera, and/or a stereo camera.

FIG. 4 is a schematic view of the rear detection proximity sensor 14 ofthe AGV 100 detecting an object 81 located behind the AGV 100. The AGV100 can be stationary or moving in reverse as indicated by referencearrow B. The object 81 is located on the storage rack 80 in a positionwhere the AGV 100 may inadvertently contact the object 81 andpotentially damage the object 81, the AGV 100, and/or the storage rack80, as well as cause any of these objects to fall over and potentiallyharm nearby operators.

The sensor head 10 is shown in an extended position by theextendable/retractable rods 35 so that the rear detection proximitysensor 14 is not obstructed by the inventory 65 that is located on theupper surface 25 of the mobile base 25. The rear detection proximitysensor 14 can be used to detect the object 81 located behind the AGV 100and thereby prevent the AGV 100 from contacting the object 81. Althoughthe object 81 is shown as inventory, the object 81 could be an operator.

FIG. 5 is a schematic view of the down facing camera 13 of the AGV 100detecting and recording one or more markers 70 attached to the inventory65 located on the upper surface 25 of the mobile base 20. The marker 70can provide inventory information, including but not limited to the typeof inventory, the quantity of inventory, and/or any instructions relatedto the inventory, such routing and/or task instructions associated withthe inventory 65 and/or the AGV 100. The sensor head 10 can be extendedand retracted by the extendable/retractable rods 35 to move the downfacing camera 13, if needed, into a position to detect and record anymarkers 70 attached to any inventory 65 located on the upper surface 25of the mobile base 20.

Referring to FIG. 5, the operator 200 can instruct the AGV 100 to go toa specific destination, retrieve a specific list of inventory, and thenperform a specific task with the inventory. The display 50 can be touchscreen so that the operator 200 can select from a list or input thedestination, the inventory, and/or the task. According to one example,the operator 200 can view and select from a list of inventory (e.g.Inventory A, Inventory B, Inventory C, Inventory D, etc.) displayed onthe display 50, as well as select a quantity of each inventory, to beretrieved. The operator 200 can also select from a list of tasks for theAGV 100 to perform, such as “Leaving when ready”. Once the AGV 100 hasarrived at the destination and the inventory has been loaded onto theAGV 100, then the AGV 100 is configured to obtain the inventoryinformation from the markers 70 attached to the inventory using the downfacing camera 13 to determine that the correct inventory has beenreceived. The inventory information captured by the down facing camera13 can be displayed on the display 50 of the AGV 100 for reference byany operator 200. Once the AGV 100 has determined that the correctinventory has been received, then AGV 100 is configured to automaticallyperform the task, such as leave to take the inventory to anotherlocation.

FIG. 6 is a schematic view of the AGV 100 following an operator 200 thatis located and moving behind the AGV 100 while the AGV 100 is moving inreverse as indicated by reference arrow C. The rear facing camera 11 isconfigured to detect and record an image of the operator 200 while theoperator 200 is stationary or moving. The image may include a shape ofthe operator 200, a pose of the operator 200, the clothing of theoperator 200, or the color of the operator 200. Based on the imagecaptured by the rear facing camera 11, the AGV 100 is configured tofollow the operator 200 in a rear follow position. Specifically, the AGV100 is configured to maintain a predetermined distance from the operator200 while the operator 200 is moving and while the AGV 100 is moving inreverse.

FIG. 7 is an exploded view of the sensor head 10 according to oneembodiment. The sensor head 10 includes a housing 22 having a top cover21. The sensor head 10 is moveable between an extended position and aretracted position by the extendable/retractable rods 35, which arecoupled at one end to the housing 22 and at an opposite end to theconsole 30. The rear facing camera 11, the side facing cameras 12, andthe down facing camera 13 are coupled to the housing 22. The reardetection proximity sensor 14 and the shelf detection proximity sensor15 are also coupled to the housing 22.

FIG. 8 is a block diagram of the AGV 100 according to one embodiment.The AGV 100 includes a controller 90 in communication with a storagedevice 97 containing data regarding map information 98, routinginformation 99, object-to-follow information 88, and inventoryinformation 89. The controller 90 is also in communication with severalmodules configured to control the operation of the AGV 100. The modulesinclude an object following module 91, an inventory checking module 92,an autonomous driving module 93, an obstacle avoidance module 94, a mapupdating module 95, and an extendable/retractable rod module 96.

The controller 90, the modules, and/or the data contained on the storagedevice 97 are configured to control the motorized wheels 40, theextendable/retractable rods 35, and/or the information displayed on thedisplay 50, all based at least in part on the information received fromthe proximity sensors 14, 15 and/or the cameras 11, 12, 13. Thecontroller 90 is configured to analyze the information received orretrieved from the cameras 11, 12, 13, the proximity sensors 14, 15, thedata on the storage device 97, and/or any of the modules and in responsecontrol the operation of the AGV 100, including the motorized wheels 40,the extendable/retractable rods 35, and/or the information displayed onthe display 50.

In one embodiment, the controller 90 is configured to receiveinformation from the shelf detection proximity sensor 14 regarding thepresence or absence of a shelf, and communicate with theextendable/retractable module 96 to determine if the location of a shelfhas been detected. If no shelf has been detected, the controller 90 isconfigured to actuate the extendable/retractable rods 35 to extend orretract the sensor head 10 until the location of the shelf is detectedby the shelf detection proximity sensor 14. Once the shelf is detected,the controller 90 is configured to stop the actuation of theextendable/retractable rods 35 to stop the sensor head 10 at thelocation of the shelf.

Information regarding the location of the shelf can be stored on thestorage device 97 and accessed by the controller 90 and/or an operatorfor future reference as needed. At any time, the controller 90 canretrieve data from the information stored on the storage device 97,including the map information 98, the routing information 99, theobject-to-follow information 88, and/or the inventory information 89 tohelp determine the location of the shelf. If the shelf and/or the AGV100 is moved to a position where the shelf detection proximity sensor 14no longer detects the shelf, the AGV 100 may continue operation and theprocess above may be repeated until the shelf or another shelf isdetected by the shelf detection proximity sensor 14.

In one embodiment, the controller 90 is configured to receiveinformation from the rear detection proximity sensor 15 (and/or the rearor side facing cameras 11, 12) regarding the presence or absence of anobject located behind the AGV 100, and communicate with the autonomousdriving module 93 and/or the obstacle avoidance module 94 to determineif an object has been detected. If no object has been detected, thecontroller 90 is configured to actuate the motorized wheels 40 to movethe AGC 100 in the desired reverse direction until an object is detectedby the rear detection proximity sensor 15. Once an object is detected,the controller 90 is configured to communicate with the autonomousdriving module 93 and/or the obstacle avoidance module 94 to determineif the AGV 100 should be stopped or moved in a different reversedirection to avoid contacting the object. The controller 90 isconfigured to actuate the motorized wheels 40 to stop the AGV 100 ormove the AGV 100 in a different reverse direction to avoid the object.

Information regarding the location of the object can be stored on thestorage device 97 and accessed by the controller 90 and/or an operatorfor future reference as needed. At any time, the controller 90 canretrieve data from the information stored on the storage device 97,including the map information 98, the routing information 99, theobject-to-follow information 88, and/or the inventory information 89 tohelp determine if an object is located behind the AGV 100 and/or whetherto stop or move in a different reverse direction to prevent the AGV 100from contacting the object. After the object has passed, the AGV 100 maycontinue operation and the process above may be repeated until anotherobject is detected by the rear detection proximity sensor 15. The objectmay be an operator, inventory, an obstacle, another AGV, or any otherobject that the AGV 100 may encounter during operation.

In one embodiment, the controller 90 is configured to receiveinformation from the rear facing camera 11, the side facing camera 12,and/or the down facing camera 13 whether one or more markers attached toan object (such as markers 70 attached to inventory 65 or markers 75attached to storage rack 80) has been detected, and communicate with theinventory checking module 92 and/or the inventory information 89 storedon the storage device 97 to identify which marker has been detected. Themarker may be attached to an object that is located behind the AGV 100,on either side of the AGV 100, or on the upper surface 25 of the mobilebase 20 of the AGV 100. Once the marker has been identified, thecontroller 90 is configured to communicate with the inventory checkingmodule 92 and/or the inventory information 89 stored on the storagedevice 97 to retrieve the inventory information and display theinventory information on the display 50 of the AGV 100. The controller90 is also configured to compare the inventory information with a listand quantity of inventory preselected by an operator to determine thatthe correct inventory has been received or detected by the AGV 100.

Information regarding the inventory can be stored on the storage device97 with the inventory information 89 and accessed by the controller 90and/or an operator for future reference as needed. At any time, thecontroller 90 can retrieve data from the information stored on thestorage device 97, including the map information 98, the routinginformation 99, the object-to-follow information 88, and/or theinventory information 89 to help detect the marker and identify theinventory information associate with the marker. The AGV 100 maycontinue operation and the process above may be repeated for any numberof markers (such as barcodes) attached to any object (such as inventoryor a storage rack storing inventory).

In one embodiment, the controller 90 is configured to receiveinformation from the rear facing camera 11 whether an object-to-followlocated behind the AGV 100 has been detected, and communicate with theobject following module 91, the autonomous driving module 93, and/or theobject-to-follow information 88 stored on the storage device 97 todetermine if an object-to-follow has been detected. Once anobject-to-follow has been detected, the controller 90 is configured tocommunicate with the object following module 91, the autonomous drivingmodule 93, and/or the object-to-follow information 88 stored on thestorage device 97 to maintain the AGV 100 within a predetermineddistance of the object-to-follow, based on images of the object asdetected and recorded by the rear facing camera 11 while the object ismoving behind the AGV 100 and while the AGV 100 is moving in reverse.The controller 90 is configured to actuate the motorized wheels 40 tomaintain the AGV 100 within the predetermined distance of theobject-to-follow.

Information regarding the object-to-follow can be stored on the storagedevice 97 with the object-to-follow information 88 and accessed by thecontroller 90 and/or an operator for future reference as needed. At anytime, the controller 90 can retrieve data from the information stored onthe storage device 97, including the map information 98, the routinginformation 99, the object-to-follow information 88, and/or theinventory information 89 to help detect the object-to-follow andmaintain the AGV 100 within the pre-determined distance. The AGV 100 maycontinue operation and the process above may be repeated for anotherobject-to-follow as detected by the rear facing camera 11. The rearfacing camera 11 can detect the object-to-follow based on images of theobject-to-follow, which may include a shape of the object, a pose of theobject, a clothing of the object, or a color of the object. The objectmay be an operator or another AGV.

In one embodiment, the controller 90 is configured to receiveinformation from any of the cameras 11, 12, 13 and sensors 14, 15, andcommunicate with the autonomous driving module 93, the obstacleavoidance module 94, the map updating module 95, the map information 98stored on the storage device 97, and/or the routing information 99stored on the storage device 97 to determine a route to follow toperform a task, while the AGV 100 is moving in reverse. Once the routehas been determined, the controller 90 is configured to communicate withthe autonomous driving module 93, the obstacle avoidance module 94, themap updating module 95, the map information 98 stored on the storagedevice 97, and/or the routing information 99 stored on the storagedevice 97 to maintain the AGV 100 on the route while the AGV 100 ismoving in reverse. The controller 90 is configured to actuate themotorized wheels 40 to maintain the AGV 100 on the route while moving inthe reverse direction.

Information regarding the route can be stored on the storage device 97with the map information 98 and/or the routing information 99 andaccessed by the controller 90 and/or an operator for future reference asneeded. At any time, the controller 90 can retrieve data from theinformation stored on the storage device 97, including the mapinformation 98, the routing information 99, the object-to-followinformation 88, and/or the inventory information 89 to help determine aroute to follow to perform a task. The AGV 100 may continue operationand the process above may be repeated for any other task.

Embodiments of the self-driving system include any combination of theabove embodiments or processes to conduct any type of task as neededusing the AGV 100.

While the foregoing is directed to embodiments of the disclosure, otherand further embodiments of the disclosure thus may be devised withoutdeparting from the basic scope thereof, and the scope thereof isdetermined by the claims that follow.

1. A self-driving system, comprising: a body having one or moremotorized wheels; and a sensor head coupled to the body and movable froma retracted position to an extended position relative to the body,wherein the sensor head comprises one or more proximity sensors.
 2. Thesystem of claim 1, wherein the body includes a console coupled in anupright positon to a mobile base, and wherein the sensor head is coupledto the console.
 3. The system of claim 1, wherein the sensor head iscoupled to the body by one or more extendable and retractable rods. 4.The system of claim 1, wherein the proximity sensors include shelfdetection proximity sensors configured to detect a location of a shelf.5. The system of claim 4, further comprising a controller coupled to thebody and configured to receive information from the shelf detectionproximity sensors, wherein the controller is configured to extend orretract the sensor head until the location of the shelf is detected bythe shelf detection proximity sensors.
 6. The system of claim 1, whereinthe proximity sensors are rear detection proximity sensors configured todetect an object located behind the body.
 7. The system of claim 6,further comprising a controller coupled to the body and configured toreceive information from the rear detection proximity sensors, whereinthe controller is configured to prevent the body from contacting theobject located behind the body based on information received from therear detection proximity sensors while the body is moving in reverse. 8.A self-driving system, comprising: a body having one or more motorizedwheels; and a sensor head coupled to the body, wherein the sensor headcomprises one or more side facing cameras, rear facing cameras, or downfacing cameras.
 9. The system of claim 8, wherein the body includes aconsole coupled in an upright positon to a mobile base, and wherein thesensor head is coupled to the console.
 10. The system of claim 8,wherein the sensor head is coupled to the body by one or more extendableand retractable rods.
 11. The system of claim 8, wherein the side facingcameras are configured to detect and record an image of one or moremarkers attached to an object located on the side of the body.
 12. Thesystem of claim 8, wherein the rear facing cameras are configured todetect and record an image of one or more markers attached to an objectlocated behind the body.
 13. The system of claim 8, wherein the downfacing cameras are configured to detect and record an image of one ormore markers attached to an object located on the body.
 14. The systemof claim 8, wherein the rear facing cameras are configured to detect andrecord an image of an object located behind the body, wherein the imageof the object includes a shape of the object, a pose of the object, aclothing of the object, or a color of the object.
 15. The system ofclaim 8, further comprising a controller coupled to the body andconfigured to receive information from the cameras.
 16. The system ofclaim 15, wherein the controller is configured to retrieve inventoryinformation and display the inventory information based on one or moremarkers attached to an object located on a side of the body as detectedand recorded by the side facing cameras.
 17. The system of claim 15,wherein the controller is configured to retrieve inventory informationand display the inventory information based on one or more markersattached to an object located behind the body as detected and recordedby the rear facing cameras.
 18. The system of claim 15, wherein thecontroller is configured to retrieve inventory information and displaythe inventory information based on one or more markers attached to anobject located on the body as detected and recorded by the down facingcameras.
 19. The system of claim 15, wherein the controller isconfigured to maintain the body within a predetermined distance of anobject based on images of the object as detected and recorded by therear facing cameras, while the object is moving behind the body andwhile the body is moving in reverse.
 20. The system of claim 19, whereinthe images of the object include a shape of the object, a pose of theobject, a clothing of the object, or a color of the object.